The invention relates generally to data communication and signal processing methods and apparatus, and in particular to a method and apparatus for reliably and efficiently communicating sequences of image data over a communications channel, for example, a 56 kilobit per second telephone communications channel.
The transmission of sequences of images, and in particular sequences of naturally occurring images such as those represented by a television signal, has been the subject of a significant amount of investigation. Typically, investigators have relied upon the highly redundant nature of successive images in the sequence and have often modeled the image data as a three-dimensional Markov process with a correlation coefficient close to unity. The three-dimensional Markov model provides a motivation for utilizing differential pulse-code-modulation (DPCM) and transform coding techniques to take account of the interframe redundancy.
By analyzing the nature of typical moving video, it is easy to become convinced that the principal change occurring between successive frames is the inhomogeneous motion of the objects within the frame. It has also been recognized that an accurate apparatus and method of estimating and compensating for this spatially dependent motion enables the construction of an interframe data compression method and apparatus which can have substantially better performance than can be achieved by sending a signal representative merely of the difference between successive frames.
As a result, various motion compensating coding methods and apparatus have been developed. These systems typically are either receiver-based motion compensation systems or transmitter-based motion compensation systems. In the receiver-based motion compensation system, the receiver makes a prediction as to the motion and compensates the previous frame for the expected motion. The transmitter, operating in the same manner, then sends only an error signal describing what must be done at the receiver in order to correct the receiver predicted frame. The error signal is typically coded in order to reduce its bandwidth.
For a transmitter-based motion compensation system, the motion estimation process occurs only at the transmitter. Displacement vectors are generally determined over various regions of the image and this data is then transmitted to the receiver along with an error information data signal. At the receiver the compensation process is performed on the previously coded image first using the motion information provided by the transmitter. The error signal data provided by the transmitter is then added to the thus compensated receiver image in order to maintain picture quality.
Unfortunately, however, both of these systems are flawed in the manner in which they are implemented. One disadvantage of receiver-based motion compensation systems is that they are, by their very nature, predictive coding systems. Because the prediction is based upon picture elements which have already been transmitted and hence received, the receiver-based motion compensation coder does not have the ability to deterministically optimize the estimate which they make of the "displacement," i.e., the motion. Thus, a receiver-based motion compensation system using a traditional raster scanning procedure, which receives a sequence wherein an object is moving rapidly toward an upper left-hand portion of the image, cannot always properly compensate therefor. Thus, when the previous frame and the local picture elements in the present frame are found to have no displacement, and the prediction method will assume no displacement for the next picture element in the raster. However, since the next picture element has in fact been displaced many picture elements toward the upper left-hand portion of the image, a very large prediction error results. A transmitter-based motion compensation system, with knowledge of the entire image, would have avoided the error by deterministically selecting the displacement estimate which minimizes the actual error.
Another major disadvantage of receiver-based motion compensation systems is that they typically employ a differential pulse-code-modulation (DPCM) representation of the error signal. A traditional DPCM system employs a coding limit of one bit per picture element, since data is transmitted for each picture element. This limitation is avoided in motion compensation systems by first segmenting the image into its moving and non-moving parts. Then, entropy coded address information is transmitted to inform the receiver of the specific regions over which an error signal will be sent. In non-moving regions, no error information need be sent. Unfortunately, however, the amount of addressing information for such a DPCM system can often exceed an allowable, reasonable limit.
The major disadvantage of transmitter-based motion compensation systems which provide good image reproduction is that they require substantial overhead data to describe the motion vector field which is typically sent in PCM form. This problem is exacerbated when it is desired to have a high resolution representation of the vector field.
It is therefore an object of the present invention to transmit sequences of images over a communications channel using relatively low bandwidth, and providing high reliability and fidelity. Other objects of the invention are a motion compensation method and apparatus which reliably provide an accurate estimate of the displacement of the pixels of a scanned image in a sequence, and an improved motion estimation method and apparatus which enable real-time, reliable, and accurate determination of regional displacement in an image transmission device.